Computer-implemented method for predicting a behavior of agents in a dynamic system with a multiplicity of interacting agents

ABSTRACT

A computer-implemented method for predicting a behavior of agents in a dynamic system with a multiplicity of interacting agents depending on the latent state thereof. For a plurality of components and for a plurality of time points up to a prediction time point, a value of a first moment of a first distribution, which models the latent state of the agents, is determined for each component. A value of a second moment of the first distribution is determined. An expected value for a first moment of a second distribution at the prediction time point is determined for each component depending on the value of the first moment of the first distribution at the prediction time point and depending on the value of the second moment of the first distribution at the prediction time point. The second distribution models the behavior of the agents depending on the latent state thereof.

CROSS REFERENCE

The present application claims the benefit under 35 U.S.C. § 119 ofGerman Patent Application No. DE 10 2022 204 723.0 filed on May 13,2022, which is expressly incorporated herein by reference in itsentirety.

FIELD

The present invention relates to a computer-implemented method forpredicting a behavior of agents in a dynamic system with a multiplicityof interacting agents.

BACKGROUND INFORMATION

Possibilities of predicting behavior in such systems are described inCharlie Tang and Russ Salakhutdinov, “Multiple Futures Prediction,”2019, NeurIPS and in Sergio Casas and Cole Gulino and Simon Suo andKatie Luo and Renjie Liao and Raquel Urtasun, “Implicit Latent VariableModel for Scene-Consistent Motion Forecasting,” 2020 ECCV.

SUMMARY

By the computer-implemented method and device according to the presentinvention, precise prediction of a behavior of agents is achieved at lowcost with regard to the required computing resources.

According to an example embodiment of the present invention, a methodfor predicting the behavior of agents in a dynamic system with amultiplicity of interacting agents depending on the latent state thereofprovides that for a plurality of components and for a plurality of timepoints up to a prediction time point, a value of a first moment of afirst distribution, which models the latent state of the agents, isdetermined for each component, wherein a value of a second moment of thefirst distribution is determined, wherein an expected value for a firstmoment of a second distribution at the prediction time point isdetermined for each component depending on the value of the first momentof the first distribution at the prediction time point and depending onthe value of the second moment of the first distribution at theprediction time point, wherein the second distribution models thebehavior of the agents depending on the latent state thereof, whereinthe expected value for the first moment of the second distributiondefines a first moment of a third distribution, wherein a second momentof the third distribution is determined for each component, wherein asum, in particular a sum weighted with at least one weight, of the thirddistributions of the component is determined, and wherein the predictionof the behavior is determined depending on the sum.

Preferably, according to an example embodiment of the present invention,it is provided that the value of the first moment of the firstdistribution is determined depending on a value of the first moment ofthe first distribution for a time point preceding the time point and onan expected value for a deterministic change of the first moment of thefirst distribution, and/or that the value of the second moment of thefirst distribution for the time point is determined depending on a valueof the second moment of the first distribution for a time pointpreceding the time point and on a covariance of the deterministic changeand on an expected value for a stochastic change of the second moment ofthe first distribution. This efficiently recursively determines therespective value.

The value of the second moment of the first distribution for the timepoint is preferably determined depending on the value of the secondmoment of the first distribution for the preceding time point and on thecovariance of the deterministic change and on a covariance of the latentstate at the preceding time point with the deterministic change and on atranspose of the covariance of the latent state at the preceding timepoint with the deterministic change and on the expected value for thestochastic change. This efficiently recursively determines the value.

Preferably, according to an example embodiment of the present invention,the expected value for the first moment of the second distribution isdetermined depending on the value of the first moment of the firstdistribution at the prediction time point. This efficiently determinesthe expected value.

Preferably, according to an example embodiment of the present invention,a covariance of the first moment of the second distribution isdetermined for each component depending on the value of the first momentof the second distribution at the prediction time point, wherein anexpected value for the second moment of the second distribution at theprediction time point is determined for each component depending on alatent state at the prediction time point, wherein the second moment ofthe third distribution is determined for each component depending on thecovariance of the first moment of the second distribution and on theexpected value for the second moment of the second distribution at theprediction time point. The method can thus be performed particularlyefficiently.

Preferably, according to an example embodiment of the present invention,a context variable is determined, which comprises an association, whichassociates at least one agent with another agent to be considered forpredicting the behavior of this agent, and/or which characterizes ahistory of the dynamic system, wherein the first moment of the firstdistribution is determined depending on the context variable, and/orwherein the second moment of the first distribution is determineddepending on the context variable, and/or wherein the expected value forthe first moment is determined depending on the context variable, and/orwherein the first moment of the third distribution is determineddepending on the context variable, and/or wherein the second moment ofthe third distribution is determined depending on the context variable,and/or wherein the at least one weight is determined for at least onecomponent depending on the context variable. A neighborhood and/or ahistory of the agents is thereby considered.

According to an example embodiment of the present invention, the historyis preferably determined depending on an observed behavior of the atleast one agent, in particular a behavior which comprises the agent'sposition or movement, wherein the agent's position or movement is inparticular acquired using a receiver for a satellite-based positiondetermination system, or wherein at least one digital image is acquired,in particular using a sensor for digital images, preferably a camera, aLiDAR sensor, ultrasonic sensor, movement sensor, thermal imagingdetector, and/or radar sensor, and the agent's position or movement isdetermined depending on at least one digital image, or wherein a signalis acquired using a speaker for receiving audible sound and the agent'sposition or movement is determined depending on the signal.

It may be provided that the context variable comprises a matrix, whoserows each represent one of the agents and whose columns each representone of the agents, wherein at least one value, in particular a binaryvalue, of an element of the matrix identified by a row and a column isdetermined and specifies whether or not the agent identified by the rowis to be considered for the prediction for the agent identified by thecolumn, or wherein at least one value, in particular a binary value, ofan element of the matrix identified by a row and a column is determinedand specifies whether or not the agent identified by the column is to beconsidered for the prediction for the agent identified by the row. Thematrix represents a neighborhood to be considered. As a result, thecalculation considers the most relevant agents in particular. As aresult, the best possible prediction is calculated particularlyefficiently.

Preferably, according to an example embodiment of the present invention,the first moment and the second moment of the first distribution aredetermined in iterations, wherein for a first one of the iterations foreach component, a value of the first moment of the first distributionand a value of the second moment of the first distribution aredetermined, which depends on the context variable. The history isthereby considered particularly efficiently.

Preferably, according to an example embodiment of the present invention,it is provided that, for the prediction, latent states of an agent aremodeled independently of one another and latent states of differentagents are modeled independently of one another, or latent states of anagent are modeled independently of one another and correspondingelements of latent states of different agents are modeled dependently onone another, or different elements of a latent state of an agent aremodeled dependently on one another and latent states of different agentsare modeled independently of one another. This makes the calculationvery efficient.

Preferably, according to an example embodiment of the present invention,at least one agent, in particular a computer-controlled machine, inparticular a robot, preferably a vehicle, a household appliance, adriven machine, a manufacturing machine, a personal assistant, or anaccess control system is controlled depending on the prediction. Thiscontrol is particularly robust.

The at least one agent may be an existing real object in the physicalworld.

According to an example embodiment of the present invention, the devicecomprises at least one processor and at least one memory, which aredesigned to perform the method. This device has advantages correspondingto those of the method.

According to an example embodiment of the present invention, a systemcomprises at least one agent, in particular a computer-controlledmachine, in particular a robot, preferably a vehicle, a householdappliance, a driven machine, a manufacturing machine, a personalassistant, or an access control system, wherein the agent or the systemcomprises the device, and wherein the device is designed to control theagent depending on the prediction. This system has advantagescorresponding to those of the method.

According to an example embodiment of the present invention, a computerprogram comprises computer-readable instructions that, when executed bya computer, cause the method to run. This computer program hasadvantages corresponding to those of the method.

Further advantageous embodiments can be taken from the followingdescription and the figures.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a schematic representation of a device for predicting abehavior of agents in a dynamic system with a multiplicity ofinteracting agents, according to an example embodiment of the presentinvention.

FIG. 2 shows a behavior of agents in an exemplary dynamic system,according to an example embodiment of the present invention.

FIG. 3 shows a prediction of the behavior of the agents in the dynamicsystem, according to an example embodiment of the present invention.

FIG. 4 shows steps in a method for predicting, according to an exampleembodiment of the present invention.

FIGS. 5A-5D show examples of neural networks, according to an exampleembodiment of the present invention.

FIG. 6 shows a schematic representation of approximations of acovariance matrix, according to an example embodiment of the presentinvention.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

FIG. 1 schematically shows a device 100 for predicting a behavior ofagents 102 in a dynamic system 104 with a multiplicity of interactingagents 102. The dynamic system 104 in the example is a physical system,in particular a technical system. The agents 102 may be physicalsystems, in particular technical systems. The agents 102 may be anexisting real objects in the physical world. The device 100 comprises atleast one processor 106 and at least one memory 108. The device 100 isdesigned to perform a below described method for predicting the behaviorof the agents 102 in the dynamic system 104. The device 100 optionallycomprises an interface 110. The agents 102 optionally comprise aninterface 112. The device 100 and the agents 102 are optionally designedto communicate via their interfaces, for example in order to transmitinformation about a behavior of the agents 102 from the agents 102 tothe device 100 or to send information about a prediction of the behaviorfrom the device 100 to the agents 102. A sensor system 114 may beprovided that is designed to acquire information about the behavior ofthe agents 102 in the dynamic system 104. The sensor system 114 may bedesigned to measure a physical property of the agents 102. The agents102 are optionally designed to provide information about their ownbehavior or about the behavior of other agents 102. For example, theinformation about their own behavior is acquired using the sensor system114. For example, the sensor system 114 is arranged in one or more ofthe agents 102 and designed to acquire the information about the ownbehavior of the respective agent 102 and/or the behavior of the otheragents 102. The sensor system 114 is, for example, designed to acquire aposition or movement of the agents 102. The sensor system 114 maycomprise a receiver for a satellite-based position determination system,e.g., a global positioning system, or a sensor for digital images, suchas a camera, a LiDAR sensor, ultrasonic sensor, movement sensor, thermalimaging detector, and/or radar sensor. The sensor system 114 is, forexample, designed to acquire a position or movement of the agents 102.The sensor system 114 may comprise a speaker for receiving audible soundand for generating audio signals. It may be provided that the sensorsystem 114 is arranged in an infrastructure 116 and is at leastintermittently connected via a communication link 118 to the interface110 of the device 100 in which the agents 102 can move. Instead of thesensor system 114, it may be provided that the data comprisesinformation about the agents 102, in particular data structured in agraph.

The agents 102 are optionally designed to determine their own behaviordepending on the prediction of the behavior of the other agents 102. Forexample, the agents 102 each comprise an actuator 120 that is designedto influence the behavior of the respective agent 102 depending on theprediction. It may also be provided that the device 100 is designed,instead of transmitting the prediction to the agents 102, to determine acontrol command for at least one agent 102 depending on the predictionand to transmit the control command to the agent(s) 102 to becontrolled. In this case, the actuator 120 is designed to execute thecontrol command. It may be provided that the device 100 is integrated inone or more of the agents 102.

Likewise provided is a computer program that contains instructions that,when executed by a computer, cause this method to run. For example, theat least one processor 102 executes the computer program.

FIG. 2 shows a behavior of agents 102 in an exemplary system 104. In theexample, the behavior of the agents 102 is observed, wherein FIG. 2shows trajectories on which the agents 102 have actually moved accordingto an observation of their behavior from a start time point of theobservation to an end time point of the observation.

For example, the dynamic system 104 is a technical system in which theagents 102 are computer-controlled machines, e.g., robots, such asvehicles, household appliances, driven machines, manufacturing machines,personal assistants, or access control systems.

The dynamic system 104 may also be another system. For example, thedynamic system 104 is a molecular dynamics in which the agents 102 areatoms or molecules whose movements are being predicted. For example, thedynamic system is a game, such as a soccer, basketball, or Americanfootball game, in which the agents are people or game equipment, e.g., aball, whose movements are being predicted.

The dynamic system 104 in the example is a roundabout 202. In theexample, the roundabout 202 has a first entry 204, a second entry 206, athird entry 208, and a fourth entry 210. In the example, the roundabout202 has a first exit 212, a second exit 214, a third exit 216, and afourth exit 218. The agents 102 in the example include vehicles. It maybe provided that the agents 102 include pedestrians. From the start timepoint, a first vehicle moves on a first observed trajectory 220 from thefirst entry 204 in the roundabout 202 and, at the end time point, islocated in the roundabout 202 in the area of the second exit 214. Fromthe start time point, a second vehicle moves on a second observedtrajectory 222 from the area of the second exit 214 in the roundabout202, exits the roundabout 202 via the third exit 216 and, at the endtime point, is located outside the roundabout 202. From the start timepoint, a third vehicle moves on a third observed trajectory 224 from thesecond entry 206 in the roundabout 202 and, at the end time point, islocated in the roundabout 202 in the area of the third exit 216. Fromthe start time point, a fourth vehicle moves on a fourth observedtrajectory 226 from an area in the roundabout 202 between the secondexit 214 and the second entry 206 in the roundabout 202 and, at the endtime point, is located in the fourth exit 218. From the start timepoint, a fifth vehicle moves on a fifth observed trajectory 228 from anarea in the roundabout 202 between the third exit 208 and the fourthentry 218 in the roundabout 202 and, at the end time point, is locatedin the first exit 212. From the start time point, a sixth vehicle moveson a sixth observed trajectory 230 in the area of the fourth entry 210until the end time point.

FIG. 3 shows a prediction of the behavior of the agents 102 in thedynamic system 104 using the example of the roundabout 202.

From the start time point, the first vehicle moves on the first observedtrajectory 220 until an observation end time point. In the example, thefirst vehicle does not move but is located in the first entry 204 untilthe observation end time point. A first predicted trajectory 320 betweenthe observation end time point and a prediction end time point isdetermined for the first vehicle. According to the prediction, the firstvehicle moves from the first entry 204 in the roundabout 202 and, at theend time point, is located in the roundabout 202 in the area of thesecond exit 214.

From the start time point until the observation end time point, thesecond vehicle moves on the second observed trajectory 222 to an area inthe roundabout 202 between the second entry 206 and the third exit 216.This portion of the second observed trajectory 222 is shown with dashedlines in FIG. 2 and in FIG. 3 . A second predicted trajectory 322between the observation end time point and the prediction end time pointis determined for the second vehicle. According to the prediction, thesecond vehicle moves from the area in the roundabout 202 between thesecond entry 206 and the third exit 216 in the roundabout 202, exits theroundabout 202 via the third exit 216 and, at the end time point, islocated outside the roundabout 202.

From the start time point until the observation end time point, thethird vehicle moves on the third observed trajectory 224. In theexample, the third vehicle does not move but is located in the secondentry 206 until the observation end time point. A third predictedtrajectory 324 between the observation end time point and the predictionend time point is determined for the third vehicle. According to theprediction, the third vehicle moves from the second entry 206 in theroundabout 202 and, at the end time point, is located in the roundabout202 in the area of the third exit 216.

From the start time point until the observation end time point, thefourth vehicle moves on the fourth observed trajectory 226 to an area inthe roundabout 202 between the third exit 216 and the third entry 208.This portion of the fourth observed trajectory 226 is shown with dashedlines in FIG. 2 and in FIG. 3 . A fourth predicted trajectory 326between the observation end time point and the prediction end time pointis determined for the fourth vehicle. According to the prediction, thefourth vehicle moves from an area in the roundabout 202 between thethird exit 216 and the third entry 208 in the roundabout 202 and, at theend time point, is located in the fourth exit 218.

From the start time point until the observation end time point, thefifth vehicle moves on the fifth observed trajectory 228 to an area inthe roundabout 202 between the fourth exit 218 and the fourth entry 210.This portion of the fifth observed trajectory 228 is shown with dashedlines in FIG. 2 and in FIG. 3 . A fifth predicted trajectory 328 betweenthe observation end time point and the prediction end time point isdetermined for the fifth vehicle. According to the prediction, the fifthvehicle moves from an area in the roundabout 202 between the fourth exit218 and the fourth entry 210 in the roundabout 202 and, at the end timepoint, is located in the first exit 212.

From the start time point until the observation end time point, thesixth vehicle moves on the sixth observed trajectory 230. In theexample, the sixth vehicle does not move but is located in the fourthentry 210 until the observation end time point. A sixth predictedtrajectory 330 between the observation end time point and the predictionend time point is determined for the sixth vehicle. According to theprediction, the sixth vehicle moves in the area of the fourth entry 210until the end time point.

The predictions, i.e., the predicted trajectories in the example, areapproximated as a Gaussian mixture distribution. The moments of theGaussian mixture distribution are determined using the method describedbelow depending on a portion of the behavior respectively observed forthe individual agents 102, i.e., in the example, the observed portion,shown in dashed lines, of the respective observed trajectory.

In the example, 95% confidence intervals are visualized for theprediction with respect to the other portion shown of the observedtrajectories.

The prediction of the trajectories, i.e., a time profile of positions ofthe agents 102, is an example. It may also be provided to determine theprediction for a distance between the agents 102, a velocity or anacceleration of the agents 102.

The behavior of the agents 102, in the example that of the vehicles, isobserved for a specified time period. The prediction is determineddepending on the behavior observed in the specified time period. In oneexample, at least one of the vehicles is an autonomous vehicle. Theprediction represents a simulation of an environment of the at least oneautonomous vehicle, wherein the at least one autonomous vehicle iscontrolled depending on the prediction.

The prediction is determined in the example by means of machine learningof a model, wherein the prediction is determined using the model.

This is described below for a latent variable X={x_(t)}_(t=0) ^(T) withx_(t)∈R^(MD) ^(x) and an observed variable Y={y_(t)}_(t=0) ^(T) ofdimension D_(y), wherein x_(t)∈R^(D) ^(x) is a set of M agents 102, andx_(t) ^(m)∈R^(MD) ^(x) is a latent state of an agent m at a time pointt, and y_(t) ^(m)∈R^(MD) ^(y) is a state of the agents 102 at the timepoint t, which is defined by

x ₀ ˜p(x ₀ |I)

x _(t) =x _(t−1)+ƒ(x _(t−1) ,I)+L(x _(t−1) ,I)w _(t−1) ,t=1, . . . ,T

y _(t) ˜N(y _(t) |g(x _(t)),QQ ^(T)(x _(t)))

wherein

-   -   x_(t) is a latent state of the agents 102,    -   x₀ is an initial value for the latent state of the agents 102 at        the start time point t=0,    -   ƒ(x_(t),I):R^(MD) ^(x) ×R^(D) ^(I) →R^(D) ^(x) is a        deterministic change in the latent state x_(t) of the agents        102, which change is modeled in the example as a neural network        parameterized with parameters θ_(ƒ),    -   L(x_(t), I):R^(MD) ^(x) ×R^(D) ^(I) →R^(D) ^(x) ^(×D) ^(x) is a        stochastic change in the latent state x_(t) of the agents 102,        which change is modeled in the example as a neural network        parameterized with parameters θ_(L), wherein θ={θ_(ƒ), θ_(L)}        denotes these parameters,    -   I∈R^(D) ^(I) is a context variable that comprises an association        N which associates each agent 102 with other agents 102 to be        considered for predicting the behavior of this agent 102, and        that comprises a history that characterizes the behavior for        each agent 102,    -   w_(t)∈R^(MD) ^(x) is a random variable from a normal        distribution w_(t)˜N(0,I) through which a disturbance variable        is introduced,    -   N(y_(t)|g(x_(t)),QQ^(T)(x_(t))) is a normal distribution whose        mean value    -   g(x_(t)):R^(MD) ^(x) →R^(MD) ^(y) ^(×MD) ^(y) is modeled by a        non-linear neural network parameterized with parameters ψ_(g),        wherein the covariance thereof    -   QQ^(T)(x_(t)): R^(MD) ^(x) →R^(MD) ^(y) ^(×MD) ^(y) is        determined by a variable Q, which is assumed to be constant or        is modeled by a non-linear neural network parameterized with        parameters ψ_(Q), wherein ψ={ψ_(g),ψ_(Q)} denotes these        parameters.

The variable Y={y_(t)}_(t=0) ^(T) comprises the states of the dynamicsystem 104, in particular the states y_(t) of the agents 102 at the timepoints t. In the example, the agents 102 are the vehicles and thevariable Y comprises the observed portions of the trajectories. Thevariable X={x_(t)}_(t=0) ^(T) comprises the latent states of the dynamicsystem 104. In the example, the variable X comprises the latent statesx_(t) of the agents 102 at the time points t. The latent states x_(t)comprise further information for a reliable prediction of a respectivefuture state y_(t+1) of the agents 102. The latent state x_(t) at thetime point t comprises, for example, the accelerations or velocities ofthe vehicles at the time point t.

The prediction is determined below for a number M of agents 102 denotedhereinafter by m.

For them, the deterministic change is

${f\left( {x_{t},I} \right)} = \begin{bmatrix}{\overset{\_}{f}\left( {x_{t}^{1},x_{t}^{N_{1}},I} \right)} \\ \vdots \\{\overset{\_}{f}\left( {x_{t}^{M},x_{t}^{N_{M}},I} \right)}\end{bmatrix}$

and the stochastic change is

${L\left( {x_{t},I} \right)} = {{diag}\begin{bmatrix}{\overset{\_}{L}\left( {x_{t}^{1},x_{t}^{N_{1}},I} \right)} \\ \vdots \\{\overset{\_}{L}\left( {x_{t}^{M},x_{t}^{N_{M}},I} \right)}\end{bmatrix}}$

wherein ƒ(x_(t) ^(m), x_(t) ^(N) ^(m) , I): R^(D) ^(x) ×R^(D) ^(x)×R^(D) ^(I) →R^(D) ^(x) denotes an update to the deterministic change,wherein L(x_(t) ^(m), x_(t) ^(N) ^(m) , I): R^(D) ^(x) ×R^(D) ^(x)×R^(D) ^(I) →R^(D) ^(x) denotes an update to the stochastic change,wherein x_(t) ^(N) ^(m) ∈R^(D) ^(x) is a message for the agent m at thetime point t, which message is determined as

x _(t) ^(N) ^(m) =AGG(x _(t),ε)^(m)

wherein N_(m)={e^(m,m′)|e^(m,m′)=1}_(m′=1) ^(M) denotes the firstinformation item N for the agent m and an operation AGG(x_(t),ε):R^(MD)^(x) ×R^(M×M)→R^(MD) ^(x) has an output m for each agent, wherein them-th agent is associated with the m-th output, wherein ε∈R^(M×M) denotesedges of a graph that define a relationship of the agents to oneanother. In the example, the relationship of the agents to one anotheris a binary value.

After t prediction steps, this model considers correlations betweenagents 102 that have a distance from one another of at most t steps. Inone example, distance means how many edges have to be followed to getfrom an agent m to an agent m′. The distance may be infinite if an agentis not connected to any other agent.

The prediction for a prediction time point T is a marginal probabilityp(y_(T)|I), which as a nested integral

p(y _(T) |I)=∫p(y _(T) |x _(T))p(x _(T) |x ₀ ,I)p(x ₀ |I)dx _(T) ,x ₀

with a probability p(y_(T)|x_(T)) a kernel p(x_(T)|x₀,I) and a Gaussianmixture model (GMM) p(x₀|I).

The kernel p(x_(T)|x₀,I) is approximated for each time step t by anormal distribution N(x_(t)|μ_(t)(I),Σ_(t)(I)) with a mean valueμ_(t)(I) and a covariance Σ_(t)(I), wherein

μ_(t)(I)=μ_(t−1)(I)+E[ƒ(x _(t−1) ,I)]

Σ_(t)(I)=Σ_(t−1)(I)+Cov[ƒ(x _(t−1) ,I)]+Cov[x _(t−1),ƒ(x _(t−1),I)]+Cov[x _(t−1),ƒ(x _(t−1) ,I)]^(T)+

E[LL ^(T)(x _(t−1) ,I)]

wherein E denotes the expected value, and Cov denotes the covariance,and wherein Cov[x_(t−1),ƒ(x_(t−1),I)] denotes the cross-covariancebetween random vectors in the arguments x_(t−1) and ƒ(x_(t−1), I).

The function ƒ(x,I) is implemented in the example as a neural network.The function L(x,I) is implemented in the example as a neural network.The function g(x) is implemented in the example as a neural network. Thefunction Q(x) is implemented in the example as a neural network.

An expected value and a covariance for an output of the respectiveneural network is determined as described, for example, in Anqi Wu,Sebastian Nowozin, Edward Meeds, Richard E. Turner, Jose MiguelHernandez-Lobato, and Alexander L. Gaunt: “Deterministic VariationalInference for Robust Bayesian Neural Networks,” in ICLR, 2019a (AnqiWu).

The cross-covariance Cov[x_(t), ƒ(x_(t), I)] is approximated, forexample, by

Cov[x _(t),ƒ(x _(t) ,I)]=Cov[x _(t) ]E[∀ _(x) _(t) ,ƒ(x _(t) ,I)]

wherein the expected value for the Jacobi matrix is approximated as inAndreas Look, Jan Peters, and Melih Kandemir: “Deterministic Inferenceof Neural Stochastic Differential Equations,” arXiv, abs/2006.08973,2020, (Andreas Look).

${E\left\lbrack {\nabla_{x_{t}}{f\left( {x_{t},I} \right)}} \right\rbrack} \approx {\prod\limits_{l = 1}^{L}{E\left\lbrack J_{t}^{l} \right\rbrack}}$

wherein J_(t) ^(l) is the Jacobi matrix in the layer l of the neuralnetwork at the time point t.

FIG. 4 shows steps in a method for the prediction p(y_(T)|I) of abehavior y_(t)={y_(t) ^(m)}_(i=1) ^(M) of agents m in the dynamic system104 with a multiplicity M of interacting agents m.

The prediction p(y_(T)|I) is determined depending on the latent statex_(t) of the agents m. The method comprises two loops, an inner loop andan outer loop.

For a first one of the iterations, the initial latent state x₀ it takenfrom a Gaussian mixture model with V components v, which is defined bythe normal distribution N(x₀|μ₀(I),Σ₀(I)).

The first moment μ_(t) and the second moment Σ_(t) of the normaldistribution N(x_(t)|μ_(t)(I),Σ_(t)(I)) is determined in the inner loopin iterations. Initially, for each component v, a value of the firstmoment μ_(0,v) and a value of the second moment Σ_(0,v) of the normaldistribution N(x_(0,v)|μ_(0,v)(I)), Σ_(0,v)(I)). In the example, thesevalues depend on the context variable I.

The values of the moments μ_(0,v) and Σ_(0,V) are determined as afunction of the context variable I by a further neural network. Anexample of this neural network with 30 fully connected layers and a tanhactivation, which is followed by a layer for the operation AGG, which isfollowed by 64 fully connected layers and a tanh activation, which isfollowed by a fully connected layer for the values of the first momentμ_(0,v) and which is followed by a further fully connected layer withexp activation, which is shown in FIG. 5 a.

The inner loop is calculated for a plurality V of components v and for aplurality of time points t at a prediction time point T. The inner loopis calculated for the prediction time point T for the plurality V ofcomponents v.

The normal distribution N(x_(t)|μ_(t)(I),Σ_(t)(I)) models the latentstate x_(t) of the agents m. The normal distributionN(y_(t)|g(x_(t)),QQ^(T)(x_(t))) models the behavior y_(t) of the agentsm depending on the latent state thereof x_(t).

In the method, a normal distribution N(a_(T,v)(I),B_(T,v)(I)) models abehavior of individual components v.

In a step 402, the context variable I is specified. The context variableI comprises, in one example, the association N^(m), which associates atleast one agent m with another agent m to be considered for predictingthe behavior of this agent m. The context variable I in the example isgiven.

The association N^(m) in one example is a matrix whose rows eachrepresent one of the agents m and whose columns each represent one ofthe agents m.

In the example, a value, in particular a binary value, is determined foreach element of the matrix.

In one example, the value of an element identified by its row and itscolumn in the matrix specifies whether or not the agent m identified bythe row is to be considered for the prediction for the agent midentified by the column.

In one example, the value of an element identified by its row and itscolumn in the matrix specifies whether or not the agent m identified bythe column is to be considered for the prediction for the agent midentified by the row.

For example, the relationships of the agents m to one another aremodeled using the graph, wherein the values ε of the edges aredetermined such that, in the graph, agents m′ neighboring an agent m areconsidered for the prediction thereof.

The context variable I comprises, in one example, the history of thedynamic system 104.

The context variable I in the example is used to determine the moments,expected values and weights, the argument of which comprises the contextvariable I.

The plurality V of components v is determined in the example with aneural network whose input variables comprise the history of the dynamicsystem 104 and the edges E from the context variable I. In one example,the history of the system 104 is defined by the observed behavior ofagents m, in particular the observed portion of the trajectories.

The edges in the example in the matrix N^(m) are binary values 0 or 1,which, for example, indicate with the value 1 that an edge existsbetween two nodes and are otherwise zero. The latent state x_(t) ^(m) ofan agent m at the time point t is represented by a node in the graph.

The trajectories are defined in one example by a temporal sequence oftwo-dimensional or three-dimensional geographic coordinates, whichindicate a temporal sequence of positions of the vehicles.

Using the operation AGG in the example, the messages x_(t) ^(N) ^(m) aredetermined depending on the matrix N^(m) and the one-dimensional inputvariable. The messages x_(t) ^(N) ^(m) are concatenated theone-dimensional input variable and mapped using the neural network ontothe values of the first moment μ_(0,v) and the value of the secondmoment Σ_(0,v).

For example, the neural network is a graph neural network. The latter isdesigned, for example, as described in Peter W. Battaglia, Jessica B.Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalez, Vinicius FloresZambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, AdamSantoro, Ryan Faulkner, Qaglar GiAlcehre, H. Francis Song, Andrew J.Ballard, Justin Gilmer, George E. Dahl, Ashish Vaswani, Kelsey R. Allen,Charles Nash, Victoria Langston, Chris Dyer, Nicolas Heess, DaanWierstra, Pushmeet Kohli, MatthewBotvinick, Oriol Vinyals, Yujia Li, andRazvan Pascanu; “Relational inductive biases, deep learning, and graphnetworks;” arXiv, abs/1806.01261, 2018.

In a step 404, for the plurality V of components v and for the pluralityof time points t=1, . . . T, in iterations, until the prediction timepoint T, for each component v, a value of the first moment μ_(t) of thenormal distribution N(x_(t)|μ_(t)(I),Σ_(t)(I)) is determined.

The value of the first moment μ_(t) is determined recursively in theexample. This means that the value of the first moment μ_(t) at a timepoint t is determined depending on a value of the first moment μ_(t−1)for a time point, e.g., t−1, preceding the time point t.

The following description is based on a tool which can be used todetermine an expected value E[f(x)] of a function f(x), a covariancematrix Cov(f(x)) of the function f(x) and a cross-covariance matrixCov(x,f(x)). For example, the expected value E[f(x)] and the covariancematrix Cov(f(x)) are determined, for example, as described in Anqi Wu.The cross-covariance matrix Cov(x,f(x)) is determined, for example, asdescribed in Andreas Look.

The tool requires that layers, the moments of which can be calculated atthe output, are used in the neural network to determine the expectedvalue E[f(x)], the covariance matrix Cov(f(x)) and the cross-covariancematrix Cov(x,f(x)). The operation AGG(x_(t),ε) is used for this purpose.

The operation AGG(x_(t),ε) is implemented, for example, as a mean valueaggregation in the respective neural network, wherein, for a layer l ofthe neural network, the message x_(t) ^(l,N) ^(m) in the time step t forthe agent m

$x_{t}^{l,N_{m}} = {\frac{1}{❘N_{m}❘}{\sum\limits_{{m\prime} \in N_{m}}x_{t}^{l,{m\prime}}}}$

is determined.

For example, for a set of messages x_(t) ^(l,N), the Kronecker productis used to determine, ⊗ depending on the E[x_(t) ^(l)] for the messagefrom a layer l, the expected value

E[x _(t) ^(l,N)]=(A└I _(D) _(x,l) )E[x _(t) ^(l)]

and the covariance

Cov[x _(t) ^(l,N)]=(A⊗I _(D) _(x,l) )Cov[x _(t) ^(l)](A⊗I _(D) _(x,l))^(T)

wherein

-   -   A∈R^(M×M) is an adjacency matrix with normalized rows that        comprise the information ε regarding the edges in matrix form,        and I_(D) _(x,l) is an identity matrix of dimension        D_(x,l)×D_(x,l). The Jacobi matrix is available as_J_(t)        ^(l)=A⊗I_(D) _(x,l) .

The tool requires that for the layers l of the neural network, the sameaffine transformation with the same weight matrix W^(l) and the samebias b^(l) is carried out. In one example, the calculation takes placefor all layers together using a Kronecker product.

E[x _(t) ^(l+1) ]=Ŵ ^(l) E[x _(t) ^(l) ]+{circumflex over (b)} ^(l)

Cov[x _(t) ^(l+1) ]=Ŵ ^(l)Cov[x _(t) ^(l)](Ŵ ^(l))^(T) +{circumflex over(b)} ^(l)

with

$x_{t}^{l} = \begin{bmatrix}x_{t}^{l,1} \\x_{t}^{l,2} \\ \vdots \\x_{t}^{l,M}\end{bmatrix}$ ${\hat{W}}^{l} = \begin{bmatrix}W^{l} & 0 & \cdots & 0 \\0 & W^{l} & \cdots & 0 \\ \vdots & {\ddots} & & \vdots \\0 & 0 & \cdots & W^{l}\end{bmatrix}$ ${\hat{b}}^{l} = \begin{bmatrix}b^{l} \\b^{l} \\ \vdots \\b^{l}\end{bmatrix}$

wherein the Jacobi matrix is available as J_(t) ^(l)=Ŵ^(l).

The value of the first moment μ_(t) is determined in the exampledepending on the expected value E[ƒ(x_(t−1),I)] for the deterministicchange ƒ(x_(t−1),I) of the first moment μ_(t).

In one example, the value of the first moment μ_(t) is determined asfollows:

μ_(t)(I)=μ_(t−1)(I)+E[ƒ(x _(t−1) ,I)]

The expected value E[ƒ(x_(t−1),I)], i.e., the change in the first momentμ_(t), in one example, is comprised depending on the edges E from thecontext variable I and the distribution of the latent state x_(t−1) atthe preceding time point, is determined as described in Anqi Wu by meansof the tool.

The edges in the example in the matrix N^(m) are binary values 0 or 1,which, for example, indicate with the value 1 that an edge existsbetween two nodes and are otherwise zero. The latent state x_(t) ^(m) ofan agent m at the time point t is represented by a node in the graph.

Using the operation AGG in the example, the messages x_(t) ^(N) ^(m) aredetermined depending on the matrix N^(m) and the latent state x_(t−1) atthe preceding time point. The messages x_(t) ^(N) ^(m) are concatenatedwith the state x_(t−1) at the preceding time point. The tool is used todetermine the expected value E[ƒ(x_(t−1), I)].

An example of a neural network ƒ(x_(t−1), I) with a layer for theoperation AGG, which is followed by 24 fully connected layers and a ReLuactivation, which is followed by a fully connected layer and a ReLuactivation, which is followed by another fully connected layer, is shownin FIG. 5 b.

In a step 406, for the plurality V of components v and for the pluralityof time points t=1, . . . T until the prediction time point T, for eachcomponent v, a value of a second moment Σ_(t) of the normal distributionN(x_(t)|μ_(t)(I), Σ_(t)(I)) is determined.

The value of the second moment Σ_(t) is determined recursively in theexample. This means that the value of the second moment Σ_(t) for thetime point t is determined depending on a value of the second momentΣ_(t−1) for a time point, e.g., t−1, preceding the time point t.

In one example, the value of the second moment Σ_(t) is determineddepending on the covariance Cov[ƒ(x_(t−1), I)] of the deterministicchange ƒ(x_(t−1), I) and on the expected value E[LL^(T)(x_(t−1), I)] forthe stochastic change L(x_(t−1), I) of the second moment Σ_(t).

It may be provided that the value of the second moment Σ_(t) for thetime point t is determined depending on the value of the second momentΣ_(t−1) for the preceding time point, e.g., t−1, and the covarianceCov[ƒ(x_(t−1), I)] of the deterministic change ƒ(x_(t−1), I) and thecovariance Cov[x_(t−1), ƒ(x_(t−1),I)] of the latent state x_(t−1) at thepreceding time point t_(t−1) with the deterministic change ƒ(x_(t−1), I)and the transpose of the covariance Cov[x_(t−1), ƒ(x_(t−1), I)] of thelatent state x_(t−1) at the preceding time point, e.g., t_(t−1), withthe deterministic change ƒ(x_(t−1), I) and the expected valueE[LL^(T)(x_(t−1), I)] for the stochastic change L(x_(t−1), I):

Σ_(t)(I)=Σ_(t−1)(I)+Cov[ƒ(x _(t−1) ,I)]+Cov[x _(t−1),ƒ(x _(t−1),I)]+Cov[x _(t−1),ƒ(x _(t−1) ,I)]^(T)+

E[LL ^(T)(x _(t−1) ,I)]

The expected value E[LL^(T)(x_(t−1), I)], i.e., the change in the secondmoment Σ_(t), in one example, is determined depending on edges ε fromthe context variable I and the latent state x_(t−1) at the precedingtime point. In the example, the tool is used to determine E[L] andCov[L] and thus E[LL^(T)(x_(t−1), I)]=Cov[L]+E[L]E[L]^(T).

The edges in the example in the matrix N^(m) are binary values 0 or 1,which, for example, indicate with the value 1 that an edge existsbetween two nodes and are otherwise zero. The latent state x_(t) ^(m) ofan agent m at the time point t is represented by a node in the graph.

Using the operation AGG in the example, the messages x_(t) ^(N) ^(m) aredetermined depending on the matrix N^(m) and the latent state x_(t−1) atthe preceding time point. The messages x_(t) ^(N) ^(m) are concatenatedwith the state x_(t−1) at the preceding time point and mapped onto theexpected value E[LL^(T)(x_(t−1), I)].

An example of a neural network L(x_(t−1),I) with a layer for theoperation AGG, which is followed by 24 fully connected layers and a ReLuactivation, which is followed by a fully connected layer and a ReLuactivation, is shown in FIG. 5 c.

The inner loop includes steps 404 and 406.

In a step 408, for each component v, an expected value E[g(x_(T,v))] forthe first moment g(x_(T,v)) of the normal distribution N(y_(t)|g(x_(t)),QQ^(T)(x_(t))) at the prediction time point T is determined. Thedistribution of y_(t), in one example, is approximated by a Gaussianmixture model (GMM) y_(T)˜Σ_(v)π(I)N(y_(T)|a_(T,v)(I),B_(T,v)(I)).

In the example, for each component v, depending on the value of thefirst moment g(x_(T,v)) at the prediction time point T, a covariance ofCov[g(x_(T,v))] of the first moment g(x_(T,v)) is determined.

In a step 410, the expected value E[g(x_(T,v))] for the first momentg(x_(T,v)) of the normal distribution N(y_(t,v)|g(x_(t,v)),QQ^(T)(x_(t,v))) is determined depending on the value of the firstmoment μ_(T,v) at the prediction time point T.

In a step 412, a first moment a_(T,v)(I) of the normal distributionN(a_(T,v)(I),B_(T,v)(I)) is determined.

In the example, for each component v, depending on a latent statex_(T,v) at the prediction time point T, an expected valueE[QQ^(T)(x_(T,v))] for the second moment QQ^(T)(x(t)) of the normaldistribution N(y_(t)|g(x_(t)), QQ^(T)(x_(t))) at the prediction timepoint T is determined as described in Anqi Wu as a function ofx_(t)˜N(x_t|μ_(t,v),Σ_(t,v)) by the tool.

In the example, the expected value E[g(x_(T,v))] defines the firstmoment a_(T,v)(I), e.g., by

a _(T,v)(I)=E[g(x _(T,v))]

The expected value E[g(x_(T,v))] is determined in the example using thetool.

An example of a neural network g(x_(t,v)) with 24 fully connected layersand a ReLu activation, which is followed by a fully connected layer, isshown in FIG. 5 d . For Q(x_(t)), a constant is assumed in the example,but more complex neural networks are also possible.

In a step 414, for each component v, a second moment B_(T,v)(I) of thenormal distribution N(a_(T,v)(I),B_(T,v)(I)) is determined.

In the example, for each component v, depending on the covarianceCov[g(x_(T,v))] of the first moment g(x_(T,v)) and on the expected valueE[QQ^(T)(x_(T,v))] for the second moment QQ^(T)(x_(T,v)) at theprediction time point T, the second moment V_(T,v)(I) of the normaldistribution N(a_(T,v)(I),B_(T,v)(I)) is determined, e.g., by

B _(T,v)(I)=COV[g(x _(T,v))]+E[QQ ^(T)(x _(T,v))]

The Covariance Cov[g(x_(T,v))] and the expected value E[QQ^(T)(x_(T,v))]are determined in the example using the tool.

The outer loop includes steps 408 to 414.

In a step 416, in particular weighted by at least one weight π_(v)(I), asum Σ_(v=1) ^(V)π_(v)(I)N(a_(T,v)(I), B_(T,v)(I)) of the third normaldistributions N(a_(T,v)(I), B_(T,v)(I)) of the components v isdetermined.

In a step 420, the prediction p(y_(T)|I) of the behavior y_(T)={y_(t)^(m)}_(t=1) ^(M) is determined depending on the sum Σ_(v=1)^(V)π_(v)(I)N(a_(T,v)(I), B_(T,v)(I)), e.g.,

p(y _(T) |I)=Σ_(v=1) ^(V)(I)N(a _(T,v)(I),B _(T,v)(I))

v=1

In a step 422, the prediction is output and/or at least one agent 102 iscontrolled depending on the prediction.

For example, the computer-controlled machine, the robot, the vehicle,the household appliance, the driven machine, the manufacturing machine,the personal assistant, or the access control system is controlled.

For example, the prediction for the molecular dynamics is determined andoutput. For example, the prediction for the movement during the game isdetermined and output.

The covariances for the different combinations of the latent states canbe processed as a matrix of dimension MD_(x)×MD_(x), which comprisesblocks which are respectively defined by one of the covariances. In oneexample, it is provided that the matrix is approximated as a thinlypopulated matrix.

FIG. 6 shows a schematic representation of approximations of acovariance matrix for five agents A, B, C, D, E.

The latent state of an agent m comprises several elements in oneexample. For example, for the trajectories, the latent state comprisesan element for a velocity of the agent m and an element for anacceleration of the agent m. The elements do not have to be physicalquantities but may also relate to other aspects of a state of an agent.

In a first approximation of the matrix, only elements from the matrixlocated on the major diagonal of the matrix are used for the prediction,wherein other elements of the matrix are not considered. This means thatthe latent states of an agent m are modeled independently of one anotherand the latent states of different agents m are also modeledindependently of one another. For example, the velocity of the agent mis modeled independently of its acceleration, and both the velocities ofthe different agents m and their accelerations are modeled independentlyof one another.

The main diagonal is shown in FIG. 6 as a solid diagonal line. In asecond approximation, the latent states of an agent are modeledindependently of one another and corresponding elements of the latentstates of different agents are modeled dependently on one another. Forexample, the velocity and the acceleration of the same agent are modeledindependently of one another, and the velocities of the different agentsare modeled dependently on one another, and the accelerations of thedifferent agents are modeled dependently of one another. This isrepresented in FIG. 6 by the solid line and diagonal, dashed lines.

In a third approximation, different elements of the latent state of anagent are modeled dependently on one another and the latent states ofdifferent agents are modeled independently of one another. This is shownin FIG. 6 by the diagonal of the shaded blocks.

The parameters θ={θ_(ƒ), θ_(L)} and ψ={ψ_(g), ψ_(Q)} of the neuralnetworks parameterized therewith are determined in the example in atraining with a data set D={Y,I}, in the example the observedtrajectories, by minimizing the expected negative logarithmicprobability:

argmin_(θ,ψ)−log E[P(y _(t) |I)]

What is claimed is:
 1. A computer-implemented method for predicting abehavior of agents in a dynamic system with a multiplicity ofinteracting agents depending on a latent state thereof, the methodcomprising the following steps: determining, for a plurality ofcomponents and for a plurality of time points up to a prediction timepoint, a value of a first moment of a first distribution, which modelsthe latent state of the agents, for each component; determining a valueof a second moment of the first distribution; determining an expectedvalue for a first moment of a second distribution at a prediction timepoint, for each component depending on the value of the first moment ofthe first distribution at the prediction time point and depending on thevalue of the second moment of the first distribution at the predictiontime point, wherein the second distribution models the behavior of theagents depending on the latent state thereof, wherein the expected valuefor the first moment of the second distribution defines a first momentof a third distribution; determining a second moment of the thirddistribution for each component; determining a sum, weighted with atleast one weight, of the third distributions of the component; anddetermining the prediction of the behavior depending on the sum.
 2. Themethod according to claim 1, wherein: the value of the first moment ofthe first distribution is determined depending on a value of the firstmoment of the first distribution for a time point preceding the timepoint and on an expected value for a deterministic change of the firstmoment of the first distribution, and/or the value of the second momentof the first distribution for the time point is determined depending ona value of the second moment of the first distribution for a time pointpreceding the time point and on a covariance of a deterministic changeand on an expected value for a stochastic change of the second moment ofthe first distribution.
 3. The method according to claim 2, wherein thevalue of the second moment of the first distribution for the time pointis determined depending on the value of the second moment of the firstdistribution for the preceding time point and on the covariance of thedeterministic change and on a covariance of the latent state at thepreceding time point with the deterministic change and on a transpose ofthe covariance of the latent state at the preceding time point with thedeterministic change and on the expected value for the stochasticchange.
 4. The method according to claim 1, wherein the expected valuefor the first moment of the second distribution is determined dependingon the value of the first moment of the first distribution at theprediction time point.
 5. The method according to claim 1, wherein acovariance of the first moment of the second distribution is determinedfor each component depending on the value of the first moment of thesecond distribution at the prediction time point, wherein an expectedvalue for the second moment of the second distribution at the predictiontime point is determined for each component depending on a latent stateat the prediction time point, wherein the second moment of the thirddistribution is determined for each component depending on thecovariance of the first moment of the second distribution and on theexpected value for the second moment of the second distribution at theprediction time point.
 6. The method according to claim 1, wherein acontext variable is determined, which includes an association whichassociates at least one agent with another agent to be considered forpredicting the behavior of this agent, and/or which characterizes ahistory of the dynamic system; and wherein: the first moment of thefirst distribution is determined depending on the context variable,and/or the second moment of the first distribution is determineddepending on the context variable, and/or the expected value for thefirst moment is determined depending on the context variable, and/or thefirst moment of the third distribution is determined depending on thecontext variable, and/or the second moment of the third distribution isdetermined depending on the context variable, and/or the at least oneweight is determined for at least one component depending on the contextvariable.
 7. The method according to claim 6, wherein: the history isdetermined depending on an observed behavior of the at least one agent,the behavior which includes the agent's position or movement, andwherein: i) the agent's position or movement is acquired using areceiver for a satellite-based position determination system, or ii) atleast one digital image is acquired using a sensor for digital images,and the agent's position or movement is determined depending on at leastone digital image, or iii) a signal is acquired using a speaker forreceiving audible sound and the agent's position or movement isdetermined depending on the signal.
 8. The method according to claim 6,wherein the context variable includes a matrix, whose rows eachrepresent one of the agents and whose columns each represent one of theagents, and wherein: i) at least one value of an element of the matrixidentified by a row and a column is determined and specifies whether ornot the agent identified by the row is to be considered for theprediction for the agent identified by the column, or ii) at least onevalue of an element of the matrix identified by a row and a column isdetermined and specifies whether or not the agent identified by thecolumn is to be considered for the prediction for the agent identifiedby the row.
 9. The method according to claim 6, wherein the first momentand the second moment of the first distribution is determined initerations, wherein for a first one of the iterations for eachcomponent, a value of the first moment of the first distribution and avalue of the second moment of the first distribution are determined,which depends on the context variable.
 10. The method according to claim1, wherein, for the prediction: i) latent states of each agent aremodeled independently of one another and latent states of differentagents are modeled independently of one another, or ii) latent states ofeach agent are modeled independently of one another and correspondingelements of latent states of different agents are modeled dependently onone another, or iii) different elements of a latent state of each agentare modeled dependently on one another and latent states of differentagents are modeled independently of one another.
 11. The methodaccording to claim 1, wherein at least one agent is controlled dependingon the prediction, the at least one agent including acomputer-controlled machine, or a robot, or a vehicle, or a householdappliance, or a driven machine, or a manufacturing machine, or apersonal assistant, or an access control system.
 12. The methodaccording to claim 1, wherein the at least one agent is an existing realobject in the physical world.
 13. A device, comprising: at least oneprocessor; and at least one memory; wherein the device is configured topredict a behavior of agents in a dynamic system with a multiplicity ofinteracting agents depending on a latent state thereof, the deviceconfigured to: determine, for a plurality of components and for aplurality of time points up to a prediction time point, a value of afirst moment of a first distribution, which models the latent state ofthe agents, for each component; determine a value of a second moment ofthe first distribution; determine an expected value for a first momentof a second distribution at a prediction time point, for each componentdepending on the value of the first moment of the first distribution atthe prediction time point and depending on the value of the secondmoment of the first distribution at the prediction time point, whereinthe second distribution models the behavior of the agents depending onthe latent state thereof, wherein the expected value for the firstmoment of the second distribution defines a first moment of a thirddistribution; determine a second moment of the third distribution foreach component; determine a sum, weighted with at least one weight, ofthe third distributions of the component; and determine the predictionof the behavior depending on the sum.
 14. A system, comprising: at leastone agent including a computer-controlled machine, or a robot, or avehicle, or a household appliance, or a driven machine, or amanufacturing machine, a personal assistant, or an access controlsystem; and a device including: at least one processor; and at least onememory; wherein the device is configured to predict a behavior of agentsin a dynamic system with a multiplicity of interacting agents dependingon a latent state thereof, the device configured to: determine, for aplurality of components and for a plurality of time points up to aprediction time point, a value of a first moment of a firstdistribution, which models the latent state of the agents, for eachcomponent; determine a value of a second moment of the firstdistribution; determine an expected value for a first moment of a seconddistribution at a prediction time point, for each component depending onthe value of the first moment of the first distribution at theprediction time point and depending on the value of the second moment ofthe first distribution at the prediction time point, wherein the seconddistribution models the behavior of the agents depending on the latentstate thereof, wherein the expected value for the first moment of thesecond distribution defines a first moment of a third distribution;determine a second moment of the third distribution for each component;determine a sum, weighted with at least one weight, of the thirddistributions of the component; and determine the prediction of thebehavior depending on the sum; wherein the device is configured tocontrol the agent depending on the prediction.
 15. A non-transitorycomputer-readable medium on which is stored a computer program includingcomputer-readable instructions for predicting a behavior of agents in adynamic system with a multiplicity of interacting agents depending on alatent state thereof, the instructions, when executed by a computer,causing the computer to perform the following steps: determining, for aplurality of components and for a plurality of time points up to aprediction time point, a value of a first moment of a firstdistribution, which models the latent state of the agents, for eachcomponent; determining a value of a second moment of the firstdistribution; determining an expected value for a first moment of asecond distribution at a prediction time point, for each componentdepending on the value of the first moment of the first distribution atthe prediction time point and depending on the value of the secondmoment of the first distribution at the prediction time point, whereinthe second distribution models the behavior of the agents depending onthe latent state thereof, wherein the expected value for the firstmoment of the second distribution defines a first moment of a thirddistribution; determining a second moment of the third distribution foreach component; determining a sum, weighted with at least one weight, ofthe third distributions of the component; and determining the predictionof the behavior depending on the sum.